Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica

BACKGROUND: The Faculty of Medical Sciences, University of the West Indies first implemented the Objective Structured Clinical Examination (OSCE) in the final MB Examination in Medicine and Therapeutics during the 2000–2001 academic year. Simultaneously, the Child Health Department initiated faculty and student training, and instituted the OSCE as an assessment instrument during the Child Health (Paediatric) clerkship in year 5. The study set out to explore student acceptance of the OSCE as part of an evaluation of the Child Health clerkship. METHODS: A self-administered questionnaire was completed by successive groups of students immediately after the OSCE at the end of each clerkship rotation. Main outcome measures were student perception of examination attributes, which included the quality of instructions and organisation, the quality of performance, authenticity and transparency of the process, and usefulness of the OSCE as an assessment instrument compared to other formats. RESULTS: There was overwhelming acceptance of the OSCE in Child Health with respect to the comprehensiveness (90%), transparency (87%), fairness (70%) and authenticity of the required tasks (58–78%). However, students felt that it was a strong anxiety-producing experience. And concerns were expressed regarding the ambiguity of some questions and inadequacy of time for expected tasks. CONCLUSION: Student feedback was invaluable in influencing faculty teaching, curriculum direction and appreciation of student opinion. Further psychometric evaluation will strengthen the development of the OSCE

Qualitative study of the impact of an authentic electronic portfolio in undergraduate medical education

Background Portfolios are increasingly used in undergraduate and postgraduate medical education. Four medical schools have collaborated with an established NHS electronic portfolio provider to develop and implement an authentic professional electronic portfolio for undergraduate students. We hypothesized that using an authentic portfolio would have significant advantages for students, particularly in familiarizing them with the tool many will continue to use for years after graduation. This paper describes the early evaluation of this undergraduate portfolio at two participating medical schools. Methods To gather data, a questionnaire survey with extensive free text comments was used at School 1, and three focus groups were held at School 2. This paper reports thematic analysis of students’ opinions expressed in the free text comments and focus groups. Results Five main themes, common across both schools were identified. These concerned the purpose, use and acceptability of the portfolio, advantages of and barriers to the use of the portfolio, and the impacts on both learning and professional identity. Conclusions An authentic portfolio mitigated some of the negative aspects of using a portfolio, and had a positive effect on students’ perception of themselves as becoming past of the profession. However, significant barriers to portfolio use remained, including a lack of understanding of the purpose of a portfolio and a perceived damaging effect on feedback

Desired Chinese medicine practitioner capabilities and professional development needs: a survey of registered practitioners in Victoria, Australia

Background The State of Victoria in Australia introduced Chinese medicine practitioner registration in 2000 and issued its education guidelines in late 2002 for introduction in 2005. This study obtained practitioners' views on desired capabilities for competent Chinese medicine practice and to identify professional development needs. Methods A questionnaire, consisting of 28 predefined capabilities in four categories with a rating scale of importance from one to five, was developed and sent to all registered Chinese medicine practitioners in the State of Victoria, Australia in October, 2005. Results Two hundreds and twenty eight completed questionnaires were returned which represented a response rate of 32.5%. Of the four categories of capabilities, technical capabilities were considered to be the most important for clinical practice. Specifically, the ability to perform acupuncture treatment and/or dispense an herbal prescription was ranked the highest. In contrast, research and information management capabilities were considered the least important. The educational background of practitioners appeared to be an important factor influencing their rating of capabilities. Significantly, nearly double the number of practitioners with Australian qualifications than practitioners trained overseas valued communication as an important capability. For continuing professional education, clinical skills courses were considered as a priority while research degree studies were not. Conclusion Registered Chinese medicine practitioners viewed skills training as important but did not support the need for research and information management training. This represents a significant hurdle to developing Chinese medicine as a form of evidence-based healthcare

'Correction:'Peer chart audits: A tool to meet Accreditation Council on Graduate Medical Education (ACGME) competency in practice-based learning and improvement

<p>Abstract</p> <p>Background</p> <p>The Accreditation Council on Graduate Medical Education (ACGME) supports chart audit as a method to track competency in Practice-Based Learning and Improvement. We examined whether peer chart audits performed by internal medicine residents were associated with improved documentation of foot care in patients with diabetes mellitus.</p> <p>Methods</p> <p>A retrospective electronic chart review was performed on 347 patients with diabetes mellitus cared for by internal medicine residents in a university-based continuity clinic from May 2003 to September 2004. Residents abstracted information pertaining to documentation of foot examinations (neurological, vascular, and skin) from the charts of patients followed by their physician peers. No formal feedback or education was provided.</p> <p>Results</p> <p>Significant improvement in the documentation of foot exams was observed over the course of the study. The percentage of patients receiving neurological, vascular, and skin exams increased by 20% (from 13% to 33%) (p = 0.001), 26% (from 45% to 71%) (p < 0.001), and 18% (51%–72%) (p = 0.005), respectively. Similarly, the proportion of patients receiving a well-documented exam which includes all three components – neurological, vascular and skin foot exam – increased over time (6% to 24%, p < 0.001).</p> <p>Conclusion</p> <p>Peer chart audits performed by residents in the absence of formal feedback were associated with improved documentation of the foot exam in patients with diabetes mellitus. Although this study suggests that peer chart audits may be an effective tool to improve practice-based learning and documentation of foot care in diabetic patients, evaluating the actual performance of clinical care was beyond the scope of this study and would be better addressed by a randomized controlled trial.</p

Evidence based post graduate training. A systematic review of reviews based on the WFME quality framework

<p>Abstract</p> <p>Background</p> <p>A framework for high quality in post graduate training has been defined by the World Federation of Medical Education (WFME). The objective of this paper is to perform a systematic review of reviews to find current evidence regarding aspects of quality of post graduate training and to organise the results following the 9 areas of the WFME framework.</p> <p>Methods</p> <p>The systematic literature review was conducted in 2009 in Medline Ovid, EMBASE, ERIC and RDRB databases from 1995 onward. The reviews were selected by two independent researchers and a quality appraisal was based on the SIGN tool.</p> <p>Results</p> <p>31 reviews met inclusion criteria. The majority of the reviews provided information about the training process (WFME area 2), the assessment of trainees (WFME area 3) and the trainees (WFME area 4). One review covered the area 8 'governance and administration'. No review was found in relation to the mission and outcomes, the evaluation of the training process and the continuous renewal (respectively areas 1, 7 and 9 of the WFME framework).</p> <p>Conclusions</p> <p>The majority of the reviews provided information about the training process, the assessment of trainees and the trainees. Indicators used for quality assessment purposes of post graduate training should be based on this evidence but further research is needed for some areas in particular to assess the quality of the training process.</p

Developing a competency-based curriculum in HIV for nursing schools in Haiti

<p>Abstract</p> <p>Background</p> <p>Preparing health workers to confront the HIV/AIDS epidemic is an urgent challenge in Haiti, where the HIV prevalence rate is 2.2% and approximately 10 100 people are taking antiretroviral treatment. There is a critical shortage of doctors in Haiti, leaving nurses as the primary care providers for much of the population. Haiti's approximately 1000 nurses play a leading role in HIV/AIDS prevention, care and treatment. However, nurses do not receive sufficient training at the pre-service level to carry out this important work.</p> <p>Methods</p> <p>To address this issue, the Ministry of Health and Population collaborated with the International Training and Education Center on HIV over a period of 12 months to create a competency-based HIV/AIDS curriculum to be integrated into the 4-year baccalaureate programme of the four national schools of nursing.</p> <p>Results</p> <p>Using a review of the international health and education literature on HIV/AIDS competencies and various models of curriculum development, a Haiti-based curriculum committee developed expected HIV/AIDS competencies for graduating nurses and then drafted related learning objectives. The committee then mapped these learning objectives to current courses in the nursing curriculum and created an 'HIV/AIDS Teaching Guide' for faculty on how to integrate and achieve these objectives within their current courses. The curriculum committee also created an 'HIV/AIDS Reference Manual' that detailed the relevant HIV/AIDS content that should be taught for each course.</p> <p>Conclusion</p> <p>All nursing students will now need to demonstrate competency in HIV/AIDS-related knowledge, skills and attitudes during periodic assessment with direct observation of the student performing authentic tasks. Faculty will have the responsibility of developing exercises to address the required objectives and creating assessment tools to demonstrate that their graduates have met the objectives. This activity brought different administrators, nurse leaders and faculty from four geographically dispersed nursing schools to collaborate on a shared goal using a process that could be easily replicated to integrate any new topic in a resource-constrained pre-service institution. It is hoped that this experience provided stakeholders with the experience, skills and motivation to strengthen other domains of the pre-service nursing curriculum, improve the synchronization of didactic and practical training and develop standardized, competency-based examinations for nursing licensure in Haiti.</p

Peer role-play and standardised patients in communication training: a comparative study on the student perspective on acceptability, realism, and perceived effect

<p>Abstract</p> <p>Background</p> <p>To assess the student perspective on acceptability, realism, and perceived effect of communication training with peer role play (RP) and standardised patients (SP).</p> <p>Methods</p> <p>69 prefinal year students from a large German medical faculty were randomly assigned to one of two groups receiving communication training with RP (N = 34) or SP (N = 35) in the course of their paediatric rotation. In both groups, training addressed major medical and communication problems encountered in the exploration and counselling of parents of sick children. Acceptability and realism of the training as well as perceived effects and applicability for future parent-physician encounters were assessed using six-point Likert scales.</p> <p>Results</p> <p>Both forms of training were highly accepted (RP 5.32 ± .41, SP 5.51 ± .44, n.s.; 6 = very good, 1 = very poor) and perceived to be highly realistic (RP 5.60 ± .38, SP 5.53 ± .36, n.s.; 6 = highly realistic, 1 = unrealistic). Regarding perceived effects, participation was seen to be significantly more worthwhile in the SP group (RP 5.17 ± .37, SP 5.50 ± .43; p < .003; 6 = totally agree, 1 = don't agree at all). Both training methods were perceived as useful for training communication skills (RP 5.01 ± .68, SP 5.34 ± .47; 6 = totally agree; 1 = don't agree at all) and were considered to be moderately applicable for future parent-physician encounters (RP 4.29 ± 1.08, SP 5.00 ± .89; 6 = well prepared, 1 = unprepared), with usefulness and applicability both being rated higher in the SP group (p < .032 and p < .009).</p> <p>Conclusions</p> <p>RP and SP represent comparably valuable tools for the training of specific communication skills from the student perspective. Both provide highly realistic training scenarios and warrant inclusion in medical curricula. Given the expense of SP, deciding which method to employ should be carefully weighed up. From the perspective of the students in our study, SP were seen as a more useful and more applicable tool than RP. We discuss the potential of RP to foster a greater empathic appreciation of the patient perspective.</p

Longitudinal Assessment of Resident Performance Using Entrustable Professional Activities

IMPORTANCE Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. OBJECTIVE To measure the development of pediatric trainees’ clinical skills over time using EPA-based assessment data. DESIGN, SETTING, AND PARTICIPANTS Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. INTERVENTIONS Longitudinal, prospective assessment using EPAs. MAIN OUTCOMES AND MEASURES Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. RESULTS Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to “unsupervised practice” varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). CONCLUSIONS AND RELEVANCE This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with post-graduation outcomes data as a means of seeking validity evidence